Top-down event-driven online workflow recognition

In this paper a framework for automatic online workflow recognition in industrial environments where the issue of concurrent activities rises, is presented. The framework consists of three main parts: The first part is devoted to detecting activity in specific Regions of Interest (ROIs) of the video sequence. This is effected by separating each frame into ROIs and representing the resulting subimages through feature vectors. By observing these vectors we can determine when there is action in a particular ROI. The second part of the framework lies in examining whether the detected activity corresponds to a workflow related event. This is accomplished by HMM modeling. Finally, the third part employs a string matching based technique to confirm the validity of the observed sequence of events or correct any detection or classification errors. This last step also addresses a top down approach by informing lower system levels (such as image representation or object tracking) about the errors committed. The performance of the proposed approach is thoroughly evaluated under real-life complex visual workflow understanding scenarios, in an industrial plant. The obtained results are compared and discussed.